Skip to content

feat:contrib/model-prism: pluggable LLM backend SPI for ADK Java via ServiceLoader#1199

Open
svetanis wants to merge 1 commit into
google:mainfrom
svetanis:feature/adk-model-prism
Open

feat:contrib/model-prism: pluggable LLM backend SPI for ADK Java via ServiceLoader#1199
svetanis wants to merge 1 commit into
google:mainfrom
svetanis:feature/adk-model-prism

Conversation

@svetanis
Copy link
Copy Markdown

@svetanis svetanis commented May 18, 2026

Add model-prism: pluggable LLM backend SPI for ADK Java via ServiceLoader

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Related discussion:

2. Description of change:

Problem:
Not every developer has access to a premium model subscription — but every
developer deserves the ability to build robust, production-grade agent systems
that scale. ADK Java already provides a strong foundation with Gemini and the
contrib/langchain4j bridge. This PR extends that foundation with a
convention-based, zero-wiring path for any OpenAI-compatible provider.

Today, using alternative providers like Groq, Ollama, or OpenRouter requires
explicit registration at startup. This PR makes that optional — providers
self-register via META-INF/services, so adding one is just a Maven
dependency.

Why mix models? Use Gemini only where its built-in tools (GoogleSearchTool,
code execution) are genuinely needed. Route heavy generation to fast free-tier
models like Groq or OpenRouter. Run lightweight steps — classification, critique,
formatting — on local Ollama at zero cost. The result is a more capable system
at a fraction of the cost of running every agent on a premium model. Individual
developers and open-source projects can build production-grade multi-agent
pipelines with no enterprise AI subscription required.

Solution:
Introduce a ServiceLoader SPI pattern (ModelProvider interface + ModelProviderRegistry) as a contrib module. Provider JARs self-register via META-INF/services — adding or removing a provider is a single Maven dependency change with zero application code modifications. A single call to ModelProviderRegistry.registerAll() in Runner or AdkWebServer.start() enables automatic discovery for all ADK applications.

This PR introduces a ServiceLoader-based SPI as a contrib module.
Adding a provider is a single Maven dependency — no application code changes.

<dependency>
  <groupId>com.google.adk.contrib</groupId>
  <artifactId>model-prism-groq</artifactId>
</dependency>

Then just use the model by name:

LlmAgent.builder().model("groq/llama3").build()

What's included

Module Purpose
model-prism-core ModelProvider SPI + OpenAiCompatibleLlm base class
model-prism-groq Groq provider (groq/.*)
model-prism-ollama Ollama provider — no API key, great for local dev
model-prism-openrouter OpenRouter provider — free tier available
model-prism-demo 11 runnable demos covering tools, streaming, web server

Tests

13 unit tests, all passing. See PR description for full list.

Manual E2E verified with Groq and Ollama.

Proposed core integration (one line)

// In Runner or AdkWebServer.start()
ModelProviderRegistry.registerAll();

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0

DefaultOpenAiMessageSerializerTest (9 tests):
  [OK] processStreamLines_textChunks_emitsPartialsThenFinal
  [OK] processStreamLines_emptyStream_emitsFinalEmptyResponse
  [OK] processStreamLines_nonDataLinesIgnored
  [OK] processStreamLines_toolCall_assemblesArgumentsAcrossChunks
  [OK] processStreamLines_multipleToolCalls_assemblesBothByIndex
  [OK] processStreamLines_errorChunk_throwsIllegalArgumentException
  [OK] processStreamLines_malformedJsonChunk_skippedGracefully
  [OK] deserializeResponse_textContent_returnsLlmResponseWithText
  [OK] deserializeResponse_toolCallContent_returnsFunctionCallPart

ModelProviderRegistryTest (4 tests):
  [OK] registerAll_singleProvider_registersPatternWithLlmRegistry
  [OK] registerAll_multipleProviders_registersAll
  [OK] registerAll_noProviders_returnsEmptyListAndNothingRegistered
  [OK] registerAll_calledTwice_registersEachTimeIndependently

Manual End-to-End (E2E) Tests:

The model-prism-demo module includes 11 runnable demo classes. To verify end-to-end:

# Set your key (OpenRouter has a free tier)
export OPENROUTER_API_KEY=your_key_here

# Basic SPI discovery
mvn -pl model-prism-demo exec:java -Dexec.mainClass=com.google.adk.models.demo.DemoApp

# Streaming
mvn -pl model-prism-demo exec:java -Dexec.mainClass=com.google.adk.models.demo.StreamingDemoApp

# Tool calling
mvn -pl model-prism-demo exec:java -Dexec.mainClass=com.google.adk.models.demo.tools.ToolsDemoApp

# ADK dev web server (open http://localhost:8080)
mvn -pl model-prism-demo exec:java -Dexec.mainClass=com.google.adk.models.demo.WebServerDemoApp

For Ollama (no API key required): install Ollama, run ollama pull llama3, then use model name ollama/llama3.

Checklist

  • I have read the CONTRIBUTING.md document.
  • My pull request contains a single commit.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

Additional context

Modules added under contrib/model-prism/:

Module Purpose
model-prism-core ModelProvider SPI, ModelProviderRegistry, OpenAiCompatibleLlm base class
model-prism-groq Groq provider (groq/.*)
model-prism-ollama Ollama provider (ollama/.*, no API key)
model-prism-openrouter OpenRouter provider (openrouter/.*, free tier available)
model-prism-demo 11 demo classes covering all major ADK features

Proposed ADK integration

  1. Three files from adk-model-prism-core would go into google-adk.jar
    • ModelProvider.java - public SPI interface
    • ModelProviderRegistry.java - ServiceLoader wiring
    • OpenAiCompatibleLlm.java - reusable base class for Open-AI-format APIs
  2. one line in Runner and AdkWebServer.start()
ModelProviderRegistry.registerAll(); // discovers all providers on the classpath

Provider JARs are published independently - no PRs to ADK core needed for new providers.

@svetanis svetanis changed the title contrib/model-prism: pluggable LLM backend SPI for ADK Java via ServiceLoader contrib/model-prism: zero-config OpenAI-compatible provider discovery for ADK Java via ServiceLoader May 19, 2026
@svetanis svetanis changed the title contrib/model-prism: zero-config OpenAI-compatible provider discovery for ADK Java via ServiceLoader feat:contrib/model-prism: zero-config OpenAI-compatible provider discovery for ADK Java via ServiceLoader May 19, 2026
@svetanis svetanis changed the title feat:contrib/model-prism: zero-config OpenAI-compatible provider discovery for ADK Java via ServiceLoader feat:contrib/model-prism: pluggable LLM backend SPI for ADK Java via ServiceLoader May 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant